Mobile robot

ABSTRACT

There is disclosed a mobile robot including an image processor that generates recognition information regarding a target object included in a taken image, and a main controller integrally controlling the robot based on this recognition information. The image processor executes steps of: generating a low-resolution image and at least one high-resolution image whose resolution higher than that of the low-resolution image; generating first target object information regarding the target object from the low-resolution image; determining which high-resolution image should be processed if two or more high-resolution images are generated, and then defining a resolution process region in the low-resolution image; processing a region in the high-resolution region corresponding to the resolution process region in the low-resolution image, so as to generate second target object information in the high-resolution image; and determining whether or not the first and the second target object information are matched; and based on this determination, using at least either of the first and the second target object information, thereby to generate the recognition information.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the foreign priority benefit under 35 U.S.C.§119 of Japanese Patent Application No. 2007-316246 filed on Dec. 6,2007, the disclosure of which is incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a robot that is autonomously mobile,processes images acquired from a visual sensor, and generatesrecognition information regarding a target object to be recognized.

2. Description of the Related Art

Conventionally, there has been disclosed an invention that processes aninput image only at a portion having features (interest portion) inhigher resolution than the other portions in the image, thereby toprovide an accurate recognition of a target object to be recognized(hereinafter referred to simply as a “target object”) in the distancewith less calculations. For example, the invention disclosed inJPH05-192895A process an image in higher resolution at a brightest pointor darkest point of the image as the interest portion.

Such a robot disclosed in JP H05-192895A defines no interest regiondepending on the robot's movement, a position of a target object such asa human or an obstacle existing around the robot, or a task performed bythe robot, which raises a problem that cannot secure a sufficientaccuracy of target object recognition. For example, the robot disclosedin JP H05-192895A may defines a region of interest on a brightest pointon the left side of the robot even when the robot turns right, whichcannot secure sufficient recognition accuracy so that a smooth movementof the robot is hindered.

The present invention provides a mobile robot that is capable ofdesirably handling various situations including movements and or tasksperformed by the robot, or target objects existing in the vicinity ofthe robot, with a higher accuracy in recognition of a target object aswell as less calculations of image processing.

SUMMARY OF THE INVENTION

The present invention provides a mobile robot including an imageprocessor that processes an image taken by a vision sensor and generatesrecognition information regarding a target object to be recognizedincluded in the taken image, and a controller that integrally controlsthe mobile robot based on the generated recognition information, and theimage processor includes: a multiple-resolution image generation unitthat down-sizes the taken image to generate a low-resolution imagehaving a resolution lower than that of the taken image and at least onehigh-resolution image having a resolution higher than that of thelow-resolution image; a low-resolution image process unit that processesthe low-resolution image generated by the multiple-resolution imagegeneration unit, and generates first target object information regardingthe target object included in the low-resolution image; an image-processdetermination unit that determines whether or not to process thehigh-resolution image in accordance with predetermined input informationor a predetermined rule, also determines which high-resolution imagehaving what resolution should be processed if the multiple-resolutionimage generation unit generates two or more high-resolution images, andthen defines a resolution process region at part of the low-resolutionimage; a high-resolution image process unit that processes a region inthe high-resolution region that corresponds to the resolution processregion at part of the low-resolution image defined by the image-processdetermination unit so as to generate second target object informationregarding the target object included in the high-resolution image; andan image-process result generation unit that determines whether or notthe first target object information generated by the low-resolutionimage process unit and the second target object information generated bythe high-resolution image process unit are met, and based on thedetermination of the first and second target object information, uses atleast either of the first and the second target object information,thereby to generate the recognition information.

Other features and advantages of the present invention will become moreapparent from the following detailed description of the invention whentaken in conjunction with the accompanying exemplary drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of a configuration of a robot controlsystem including a robot or robots, according to an embodiment of thepresent invention.

FIG. 2 is a schematic diagram showing an example of a self-positiondetection and an object detection executed by a robot according to theembodiment of the present invention.

FIG. 3 is a drawing showing an example of a local map used in the robotcontrol system in FIG. 1

FIG. 4 shows an example of task information database stored on storageof a management computer in FIG. 1.

FIG. 5 shows an example of a task schedule table stored on the storageof the management computer in FIG. 1.

FIG. 6 is a block diagram showing a configuration of the robot accordingto the embodiment of the present invention.

FIG. 7 is a drawing showing an example of extracting face regions from alow-resolution image, according to the embodiment of the presentinvention.

FIG. 8 is a drawing explaining how to acquire information indicating inwhich area a person exists based on the radio filed intensity, accordingto the embodiment of the present invention.

FIG. 9 is a drawing explaining an example of how a transport task isexecuted by the robot according to the embodiment of the presentinvention.

FIG. 10 is a block diagram showing a configuration of a main controllerof the robot in FIG. 6.

FIG. 11 is a flow chart showing various steps performed by an imageprocessor in FIG. 6.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, with reference to the attached drawings, descriptions willbe provided on an embodiment that implements a mobile robot (hereinafterreferred to simply as a “robot”) of the present invention. First, withdereference to FIG. 1, descriptions will be given on a generalconfiguration of a robot control system A including the robot Raccording to the embodiment of the present invention. FIG. 1 shows aschematic diagram of the robot control system A including the robot Raccording to the embodiment of the present invention.

[Robot Control System]

As shown in FIG. 1, the robot control system A includes at least onerobot R, a station 1 that is connected to the robot R via wirelesscommunication, a management computer 3 connected to the station 1 via arobot network 2, and a terminal 5 that is connected to the managementcomputer 3.

In FIG. 1, the robot control system A includes plural robots Ra, Rb, Rc(hereinafter referred to simply as the “robot R” unless otherwisestated), and each robot R executes a task in accordance with anexecution plan of the task (task schedule) that is predefined for eachrobot R through the management computer 3.

Hereinafter, a two-leg type autonomous mobile robot will be described asan example of the invention.

The robot R executes a task in response to an execution instructioninput from the management computer 3, and at least one robot R islocated within a task execution area predefined as an area where therobot R executes the task.

In the case of FIG. 1, three robots R are illustrated: the robot Ra isexecuting a task to guide a visitor to a particular place such as ameeting room (guide task); the robot Rb is executing a task of carryingan article to a particular person (transportation task); and the robotRc stays in the stand-by mode until a new task is given to.

As shown in FIG. 2, the robot R includes a head R1, arms R2, legs R3, abody R4 and a back housing section R5. The head R1, each arm R2 and eachleg R3 are connected to the body R4, each of which is driven byrespective actuators (driving means), and the robot R's bipedal walk iscontrolled by an autonomous motion control unit 50 (see FIG. 6). Moredetails on such a robot's bipedal walk mechanism are described in JP2001-62760A, for example.

When executing a guide task, for example, the robot R guides a person Hin a predetermined guide area (e.g. movement area such as an office or ahallway). In this example, the robot R irradiates light (e.g. infraredray, ultraviolet ray, leaser beam) and radio waves toward acircumference of the robot R, thereby to detect the person H wearing atag T in the circumferential region, identify a position of the detectedperson H and approach to him or her, so that the robot R executes apersonal identification to find who the person H is, based on the tag H.This tag T receives infrared ray and radio waves transmitted from therobot R for the sake of identifying the position (distance andorientation) of the person H. Based on signals regarding alight-receiving orientation included in the received infrared ray andthe robot ID included in the received radio waves, the tag T generates areceiving report signal that includes the tag ID number and sends thisreceiving report signal back to the robot R. When receiving thereceiving repot signal, the robot R recognizes the distance andorientation to the person H wearing the tag T, so that the robot R canapproach this person H.

When the robot R autonomously moves within the guide area to execute aparticular task (e.g. guide task or transport task), the robot Rirradiates laser slit light or infrared ray, thereby to detect groundconditions or find marks provided on the ground (or floor) surface.Specifically, the robot R determines where itself moves within themovement area, and if the robot R determines itself moving within aregular movement area, the robot R irradiates laser slit ray onto theground surface to detect steps, rolls or obstacles on the ground (orfloor). If the robot R detects itself moving within an M-marked area,the robot R irradiates the infrared ray onto the ground surface todetect a mark M, so that the robot R recognizes and corrects itsposition and the like. The mark M may be, for example, made ofreflective material that recursively reflects the infrared ray. The markM includes position data, and this position data is virtually includedin map data, and actually stored on the storage 30 (see FIG. 6). The mapdata includes the position data regarding the mark M provided on apredetermined position within the guide area, as well as data regardingan M-marked area, which has a predetermined broader range in addition tothe mark M position. The M marked area refers to an area defined with apredetermined distance from the mark M; for example, a circular areawith a radius of 1 to 3 m from the mark M as a center; a rectangulararea extending 3 m (toward the robot R side) from the mark M, or thelike.

Returning to FIG. 1, descriptions will be given on the configuration ofthe robot control system A.

The station 1 relays data exchange between the robot R and themanagement computer 3. Specifically, the station 1 sends an executioninstruction output from the management computer 3 to the robot R, andalso receives data regarding the robot R's conditions and a signalrepresenting that the robot R has received the execution instruction(i.e. receiving report signal) from the robot R, and then outputs thesignal received to the management computer 3.

As for the station 1, at least one station 1 may be provided in eachtask execution area to ensure a stable data exchange between the robot Rand the management computer 3.

In some case, a task execution area may be located across multiplefloors in the building. In such a case, the task execution area maypreferably be divided into multiple sub-areas, so that each floor mayhave a single sub-area. If a single station 1 cannot cover the wholetask area, multiple stations 1 may be provided across the task area.

The station 1, the management computer 3 and the network 4 are connectedwith one another through the robot network 2, which may be implementedusing LAN (Local Area Network), for example.

The management computer 3 manages at least one robot R, and providesvarious controls on the robot R's actions, movements, speech etc., viathe station 1 and the robot network 2, as well as providing informationnecessary for the robot R. Note that this necessary information mayincludes a detected person's name and maps in the vicinity of the robotR, etc, and such information is stored on the storage 3 a of themanagement computer 3.

FIG. 3 is a drawing that shows an example of a local map used in therobot control system A of FIG. 1. In this example, the guide area 301 isa rectangular area on a particular floor in a building, as seen in FIG.3A. The robot R and a person H guided by the robot R enter through ahallway 303 outside a door way 302 of a guide area 301 into this guidearea 301. Inside the door way 302, the hall 304 stretches out, and thereis an information counter 305 at the back corner of the hall 304, andthere are multiple meeting rooms 306 (306 a, 306 b, 306 c) partitionedas a separate room along the wall side of the guide area 301. Theinformation counter 305 includes an L-shaped counter table 305 a and acounter space 305 b where a clerk serves. There is provided the station1 in the counter space 305 b. The management computer 3 stores on thestorage 3 a (see FIG. 1), local maps (local map data) that associatelocational information regarding hallways and rooms, etc. withcorresponding positional coordinates, as well as a global map that ismap information regarding task execution areas build up with the abovelocal maps.

The management computer 3 also stores on the storage 3 a (see FIG. 1)the task information database 400 storing information regarding tasksexecuted by the robot R (task data).

As shown in FIG. 4, the task information database 400 includes variousinformation items, such as: a task ID as an identifier uniquely assignedto each task; priority of a task; importance of a task; a robot ID as anidentifier uniquely assigned to each robot, which is used wheninstructing the robot to execute a task; task content representing suchas “guide”, “transport (transport of an article)”, etc.; a start pointto start a task in the task execution area; an end point to end a taskin the task execution area; time required for executing the task; ascheduled start time to start a task (start time); a scheduled end timeto end a task (end time); and status of a task.

The management computer 3 also assigns to each robot R a task executionplan (task schedule) that schedules the robot R to execute a task.

As shown in FIG. 5, the task schedule table 500 includes variousinformation items, such as: priority order of tasks to be executed bythe robot R; a task ID to identify a specific task included in the taskinformation database 400 (see FIG. 4); priority of a task; content of atask; and status of a task.

The task schedule table 500 includes these information items arrangedfor each robot R, so as to understand what task is assigned to whichrobot R in what priority order.

Returned to FIG. 1, descriptions will be provided on the configurationof the robot control system A.

The terminal 5 is connected to the management computer 3 via the network4, registers information regarding persons, and or modifies theregistered information on the storage 3 a. The terminal 5 also registersa task to be executed by the robot R, changes a task schedule that isdefined on the management computer 3, and inputs instructions regardingthe robot R's actions.

[Robot]

Hereinafter, descriptions will be provided on the robot R.

The robot R includes cameras C, C, a speaker S, microphones MC, MC, animage processor 10, an audio processor 20, a storage 30, a maincontroller (also referred to as a “controller”) 40, an autonomous motioncontrol unit 50, a wireless communication unit 60, a battery 70, anobject detector 80 and a circumference sensor 90, as well as the headR1, each arm R2, each leg R3, the body R4 and the back housing sectionR5.

The robot R further includes a gyro sensor SR1 that detects anorientation in which the robot R heads, and a GPS (Global PositioningSystem) receiver SR2 for acquiring positional coordinates thatidentifies a current position of the robot R on a predetermined map.

[Cameras]

The cameras (also referred to as a “vision sensor”) C, C, capturedigital data on images in the proceeding direction ahead of the robot R,and a color CCD (Charge-coupled Device) may be used as the cameras C, C.The cameras C, C are disposed on the right and left sides in pair at thesame height level, and output captured images to the image processor 10.The cameras C, C, the speaker S and the microphones MC, MC are providedin the head R1. The speaker S (also referred to as an “audio outputmeans”) utters predetermined voices synthesized in the audio processor20.

[Image Processor]

The image processor 10 processes images taken by the cameras C, C, andrecognizes a target object, such as an obstacle or a person in thevicinity of the robot R, based on the taken images, so as to graspcircumstantial conditions of the robot R. The image processor 10includes a multiple-resolution image generation unit 11, alow-resolution image process unit 12, an image-process determinationunit 13, a high-resolution image process unit 14, and an image-processresult generation unit 15.

The multiple-resolution image generation unit 11 downsizes an imagetaken by the camera C and generates a low-resolution image whoseresolution is lower than that of the original image taken by the cameraC (also referred to simply as an “original image” or a “taken image”),and at least one high-resolution image whose resolution is higher thanthat of this low-resolution image. Specifically, when downsizing animage, the multiple-resolution image generation unit 11 corrects adistortion of the image taken by the camera C and downsizes the image byaveraging 2×2 pixels into a single pixel, or removing pixels atspecified intervals. The multiple-resolution image generation unit 11outputs a low-resolution image to the low-resolution image process unit12, and a high-resolution image to the image-process determination unit13, as well.

For example, if each camera C takes an image with 1280×960 pixels, themultiple-resolution image generation unit 11 generates two differentimages having different resolutions respectively, i.e., a low-resolutionimage downsized to 640×480 pixels and a high-resolution image of theoriginal image.

The multiple-resolution image generation unit 11 may also generate threeor more different images having different resolutions, respectively. Forexample, the multiple-resolution image generation unit 11, using theoriginal image, may generate three different images: a low-resolutionimage downsized to 640×480 pixels, a high-resolution image (medium)downsized to 800×600 pixels and a high-resolution image (high) that isthe original image.

Note that a target object may be a person, an obstacle, a tag attachedon an object to be detected, or a transport container on which articlesare placed. Target object information includes various informationregarding a target object, such as type of a target object, orientationof a target object, distance to a target object, face region of aperson, and region of a target object in an image.

The low-resolution image process unit 12 processes a low-resolutionimage generated by the multiple resolution image generation unit 11, andgenerates target object information regarding a target object in thelow-resolution image (also referred to as a “first target objectinformation”). Note that the low-resolution image process unit 12processes an image having the lowest resolution among images generatedby the multiple-resolution image generation unit 11.

In addition, the low-resolution image process unit 12 generates a pairof low-resolution images from two original images from the cameras C, Con the right and left. The low-resolution image process unit 12 alsocalculates a parallax between each pair of pixels included in thelow-resolution images in pair based on the block correlation, so as togenerate a parallax image from the low-resolution images in pair. Notethat this calculated parallax represents a distance from the Robot R tothe target object whose images are taken by the cameras C, C. Ifinformation regarding a distance to the target object is not necessary,the low-resolution image process unit 12 may process only either of thelow-resolution images in pair.

The low-resolution image process unit 12 stores several frames of pastimages to extract a moving target object, and calculates a displacementof each pixel across those frames of the past images by using apredetermined correlation algorithm or the like, thereby to generate amotion image. Based on the above parallax and motion images, if thereare pixels having greater displacements in a predetermined distancerange from the cameras C, C, the low-resolution image process unit 12estimates that there is a person, and extracts part of the parallax onlydefined in the predetermined distance range from the cameras C, C, as amoving target object.

The low-resolution image process unit 12 determines a head region in themoving target object extracted from the low-resolution image, from whicha face region and a face position are identified. Similarly, a handposition may further be identified based on a size and or a shape atpart of the extracted moving target object. Details of how to determinea person's face region is disclosed in JP2006-171929A, for example.

Next, the low-resolution image process unit 12 extracts an obstacle(target object) in such a manner that the process unit 12 generates anedge-detected image by detecting edges in the low-resolution image, orgenerates a binary image by converting each pixel into a white or blackcolor based on the density values of the image. In addition, if therobot R transports an article (i.e. executes a transport task), thelow-resolution image process unit 12 extracts a transport container onwhich the article is placed from the low-resolution image.

The low-resolution image process unit 12 outputs data regarding the faceregion of the person or the transport container as extracted above, asinput information, to the image-process determination unit 13, and alsooutputs target object information generated based on this extract datato the image-process result generation unit 15.

The image-process determination unit 13 determines whether or not toprocess a high-resolution image in accordance with the predeterminedinput information or the predetermined rule, also determines whichhigh-resolution image having what resolution should be processed if twoor more high-resolution images with different resolutions have beengenerated by the multiple-resolution image generation unit 11, anddefines a resolution process region at part of the low-resolution image.The image-process determination unit 13 outputs the high-resolutionimage having the resolution determined to be processed and theresolution process region as defined above to the high-resolution imageprocess unit 14. Meanwhile, if the image-process determination unit 13determines not to process any high-resolution image, the process unit 13outputs this determined result to the image-process result generationunit 15.

Hereinafter, specific descriptions will be provided on theabove-mentioned determination and the definition of the resolutionprocess region by the image-process determination unit 13 according tothe embodiment of the present invention, with reference to the followingvariations 1 to 11. Note that, in the following examples, descriptionswill be omitted on a sound-source position determination unit 21 c, amain controller 40, an internal condition detection unit 45, a behaviorplan management unit 46 and an object detector 80 and others, but willbe described later on.

<Variation 1: Defining Resolution Process Region at Center>

The image-process determination unit 13 defines the resolution processregion at a center of the low-resolution image in accordance with apredetermined rule. In this example, the image-process determinationunit 13 defines, for example, a rectangular resolution process regionwith 200 (horizontal)×100 (vertical) pixels at the center of thelow-resolution image.

In this case, no input information is input; therefore, theimage-process determination unit 13 defines the resolution processregion in accordance with the following rules: for example, theimage-process determination unit 13 keeps the resolution process regionas defined while the robot R is operating. Alternatively, theimage-process determination unit 13 may define the resolution processregion only when the robot R moves, and when the robot R stops, thedefined resolution process region may be canceled. Furtheralternatively, the resolution process region is defined once, andthereafter, this resolution process region defined may be canceled if notarget object is recognized in a predetermined time period. Theimage-process determination unit 13 determines to process ahigh-resolution image when the resolution process region is defined, anddetermines not to process a resolution process region when the definedresolution process region is canceled.

The resolution process region is not limited to a specific size (area)and shape, and may be formed in a rectangle, circular or horizontallyellipsoid shape.

<Variation 2: Repeatedly Defining Resolution Process Region inHorizontal Direction>

The image-process determination unit 13 repeatedly defines theresolution process region, shifting this region in the horizontaldirection in accordance with a predetermined rule. In this example,first, the image-process determination unit 13 defines the resolutionprocess region at a center of the previous frame of the low-resolutionimage, and then horizontally shifts the region toward the right or leftside in the next frame of the low-resolution image by the predeterminednumber of pixels. The image-process determination unit 13 repeats thisstep until the resolution process region reaches the right or left endof the low-resolution image. When the resolution process region reachesthe right or left end of the low-resolution image, the image-processdetermination unit 13 may once again define the resolution processregion at the center of the low-resolution image, and then re-shift thisregion toward the reverse direction until the region reaches the otherend of the image.

Alternatively, the image-process determination unit 13 may define theresolution process region at the right or left end of the low-resolutionimage in advance, and then shift the resolution process region towardthe other end of the image.

The resolution process region is not limited to a specific size andshape, and may be formed in a rectangle, circular or horizontallyellipsoid shape.

<Variation 3: Defining Resolution Process Region on Face Region ofPerson>

The image-process determination unit 13 may define the resolutionprocess region on an extracted face region of a person extracted by thelow-resolution image process unit 12. FIG. 7 shows an example of faceregions extracted from the low-resolution image by the low-resolutionimage process unit 12. In this example, the low-resolution image processunit 12 generates a face region image including four face regionsextracted. When the generated face region images are input to theimage-process determination unit 13, then the image-processdetermination unit 13 defines these face regions to be the resolutionprocess region (see non-meshed portions in FIG. 8).

The image-process determination unit 13 may define more than oneresolution process region in the low-resolution image.

<Variation 4: Defining Resolution Process Region Depending on FutureTraveling Direction>

The image-process determination unit 13 defines the resolution processregion depending on the future traveling direction. In this example, ifthe robot R's future traveling direction is straightforward; that is,the robot R is scheduled to advance straightforward from now, theimage-process determination unit 13 defines the resolution processregion at the center of the low-resolution image. Then, when the robotR's future traveling direction is toward the right side; that is, therobot R is scheduled to turn to the right from now, the image-processdetermination unit 13 defines the resolution process region on the rightside of the low-resolution image. When the robot R's future travelingdirection is toward the left side this time; that is, the robot R isscheduled to turn to the left from now, the image-process determinationunit 13 defines the resolution process region on the left side of thelow-resolution image.

If there is only a small difference in angle between the direction inwhich the robot R currently faces and the future traveling direction inwhich the robot R will travel, that is, if the robot R's turning angleis relatively small, the image-process determination unit 13 shifts theresolution process region from the center of the low-resolution image alittle toward the traveling direction; or otherwise, if the robot R'sturning angle becomes relatively great, the image-process determinationunit 13 defines the resolution process region to get closer toward aneither (right or left) end of the low-resolution image in the travelingdirection.

If the robot R turns so greatly that the robot R's traveling directionmay deviate out of the photographable range of the cameras C, C. In sucha case, the image-process determination unit 13 defines the resolutionprocess region at either (right or left) end of the low-resolution imagein the future traveling direction. Defining the resolution processregion in such a way, simply turning the robot R's head R1 to thetraveling direction enables a rapid recognition of a target object inthe future traveling direction, as well as reducing at minimum theturning rate of the robot R itself when recognizing the target object.

In this case, the behavior plan management unit 46 outputs to theimage-process determination unit 13 the robot R's future travelingdirection as input information based on the robot R's behavior planmanaged by the unit 46. Alternatively, the autonomous motion controlunit 50 may output to the image-process determination unit 13 the inputtraveling direction of the robot R as input information.

As shown in FIG. 4, the main controller 40 acquires a task that iscurrently being executed from the task information database 400 of themanagement computer 3, and obtains a future traveling direction of therobot R. The main controller 40 determines, for example, that adirection in which the end point included in the task is viewed from therobot R's current point should be the robot R's future travelingdirection, and outputs this determined direction as input information tothe image-process determination unit 13.

The information regarding the robot R's current orientation is input viathe main controller 40 from the gyro sensor SR1, and the informationregarding the robot R's current point is input via the main controller40 from the GPS receiver SR2.

<Variation 5: Defining Resolution Process Region Depending on FutureTraveling Speed>

The image-process determination unit 13 may be configured to be morelikely to determine to process a higher-resolution image and narrow theresolution process region as the input traveling speed or the futuretraveling speed becomes faster. Such a determination more likely toprocess a higher-resolution image as the input traveling speed or thefuture traveling speed becomes faster is for the sake of enhancingaccuracy in recognition of the target object in the distance. In thiscase, the resolution process region is defined depending on the robotR's future traveling direction.

If the multiple-resolution image generation unit 11 generates twodifferent images, the image-process determination unit 13 may determinein such a manner:

-   not to process a high-resolution image while “stopping”-   to process a high-resolution image while “walking”-   to process a high-resolution image while “running”    If the multiple-resolution image generation unit 11 generates three    different images, the image-process determination unit 13 may    determine in such a manner:-   not to process a high-resolution image while “stopping”-   to process a high-resolution image (medium) while “walking”-   to process a high-resolution image (high) while “running”    In this case, the image-process determination unit 13 defines the    resolution process region to be a rectangle of 200 (horizontal) ×100    (vertical) pixels while the robot R is walking, and to be a    rectangle of 100 (horizontal) ×50 (vertical) pixels while the robot    R is running. Although processing a high-resolution image, since the    resolution process region is defined to be narrower, thereby to    reduce the calculations by the high-resolution image process unit    14.

Specific examples of the robot R's future traveling speed will be shownas follows, and this is the same as the case of the input travelingspeed:

-   -   Future traveling speed (input traveling speed)=0(m/s) while        “stopping”    -   0(m/s)<future traveling speed (input traveling speed)≦3(m/s)        while “walking”    -   Future traveling speed (input traveling speed)>3(m/s) while        “running”

The behavior plan management unit 46 outputs to the image-processdetermination unit 13 the robot R's future traveling speed from thebehavior plans managed by the unit 13 as input information.Alternatively, the autonomous motion control unit 50 may output to theunit 13 the input traveling speed of the robot R as input information.

In this case, as shown in FIG. 4, the main controller 40 acquires a taskcurrently being executed from the task information database of themanagement computer 3, and obtains the future traveling speed of therobot R. For example, the main controller 40 finds a traveling distancebased on a difference in distance between the start point and the endpoint included in the task, and outputs a value obtained by dividing thefound traveling distance by the required time as input information(future traveling speed) to the image-process determination unit 13.

<Variation 6: Defining Resolution Process Region Depending on Directionof Sound Source>

The image-process determination unit 13 defines the resolution processregion depending on the direction of a sound source that is input. Ifthe direction of the sound source is located straight ahead of the headR1 of the robot R, the image-process determination unit 13 defines theresolution process region at the center of the low-resolution image. Ifthe direction of the sound source is located on the right side of thehead R1, the image-process determination unit 13 defines the resolutionprocess region on the right side of the low-resolution image. If thedirection of the sound source is located on the left side of the headR1, the image-process determination unit 13 defines the resolutionprocess region on the left side of the low-resolution image.

If there is a small difference in angle between the direction where thehead R1 heads and the direction of the sound source, the image-processdetermination unit 13 shifts the resolution process region from thecenter of the low-resolution image a little toward the direction of thesound source. Otherwise, if there is a greater difference in between thedirection where the head R1 heads and the direction of the sound source,the image-process determination unit 13 defines the resolution processregion to get closer toward an either (right or left) end of thelow-resolution image in direction of the sound source.

In some case, the sound source may be located on the back side of therobot R, and deviate out of the photographable range of the cameras C,C. In such a case, the image-process determination unit 13 defines theresolution process region at either (right or left) end of thelow-resolution image in the future traveling direction. Defining theresolution process region in such a way, simply turning the head R1 ofthe robot R to the direction of the sound source enables a rapidrecognition of the target object in the direction of the sound source,as well as making at minimum the turning rate of the robot R itself whenrecognizing the target object.

The main controller 40 outputs to the image-process determination unit13 the direction and the sound pressure of the sound source that areidentified by the sound-source position determination unit 21 c as inputinformation. In order to obtain the head R1's current heading direction,information regarding the head R1's current heading direction acquiredfrom the autonomous motion control unit 50 can be added to the abovementioned information regarding the robot R's current heading direction.

<Variation 7: Changing Resolution Depending on Sound Pressure>

The image-process determination unit 13 may be configured to be morelikely to determine to process a higher-resolution image and narrow theresolution process region as the input sound pressure becomes smaller.This determination relies on an assumption that a smaller sound pressuremeans a longer distance to a sound source of a target object. In thiscase, the resolution process region may be defined depending on thedirection of the sound source.

If the multiple-resolution image generation unit 11 generates twodifferent images, the image-process determination unit 13 determines asfollows:

-   Low sound pressure: to process a high-resolution image-   Intermediate sound pressure: to process a high-resolution image-   High sound pressure: not to process a high-resolution image    If the multiple-resolution image generation unit 11 generates three    different images, the image-process determination unit 13 determines    as follows:-   Low sound pressure: to process a high-resolution image (high)-   Intermediate sound pressure: to process a high-resolution image    (medium)-   High sound pressure: not to process a high-resolution image

The image-process determination unit 13 defines the resolution processregion to be a rectangle of 200 (horizontal)×100(vertical) pixels if thesound pressure is low; and defines the resolution process region to be arectangle of 100 (horizontal)×50(vertical) pixels if the sound pressureis medium. Examples of various sound levels are shown as follows:

-   Low sound pressure: sound pressure≦20 (dB)-   Medium sound pressure: 20 (dB)<sound pressure≦60 (dB)-   High sound pressure: sound pressure>60 (dB)    <Variation 8: Defining Resolution Depending on Direction of Tag>

The image-process determination unit 13 defines the resolution processregion depending on the input direction of a tag T. As shown in FIG. 8,when a tag attached on an object to be detected (person H) is locatedstraight ahead of the head R1 of the robot R, the image-processdetermination unit 13 defines the resolution process region at thecenter of the low-resolution image; if the direction of the tag T islocated on the right side of the head R1, the image-processdetermination unit 13 defines the resolution process region on the rightside of the low-resolution image; and if the direction of the tag T islocated on the left side of the head R1, the image-process determinationunit 13 defines the resolution process region on the left side of thelow-resolution image.

If there is a small difference in angle between the direction in whichthe head R1 faces and the direction of the tag T, the image-processdetermination unit 13 shifts the resolution process region from thecenter of the low-resolution image a little toward the direction of thetag T. Otherwise, if there is a relatively great difference in anglebetween the robot R's current facing direction and the direction to thetag, the image-process determination unit 13 defines the resolutionprocess region to get closer toward an either (right or left) end of thelow-resolution image in the direction of the tag T.

If the person H wearing the tag T is located behind of the robot R orthe like, the tag T may deviate out of the photographable range of thecameras C, C. In such a case, the image-process determination unit 13defines the resolution process region at either (right or left) end ofthe low-resolution image in direction of the tag T side. Defining theresolution process region in such a way, simply turning the robot R'shead R1 toward the direction of the tag T enables a rapid recognition ofthe person H in direction of the tag T, as well as making the turningrate of the robot R itself at minimum when recognizing the person H.

The main controller 40 outputs to image-process determination unit 13the direction and the distance to the tag T detected by the objectdetector 80 as input information.

<Variation 9: Changing Resolution Depending on Distance to Tag>

The image-process determination unit 13 may be configured to be morelikely to determine to process a higher-resolution image and narrow theresolution process region as the distance to the tag T that is inputbecomes greater. As shown in FIG. 8, the vicinity area of the robot R isclassified into four areas based on the distance from the robot R.Specifically, in the order of shorter to longer distance from the robotR, the vicinity area is classified into the area 1, the area 2, the area3 and the area 4. These areas are previously related to the respectiveradio filed intensities (i.e. distances to the tag T detected by theobject detector 80). In this case, the resolution process region isdefined depending on the direction of the tag T.

If the multiple-resolution image generation unit 11 generates twodifferent images, the image-process determination unit 13 determines asfollows:

-   Area 1: not to process a high-resolution image-   Area 2: not to process a high-resolution image-   Area 3: to process a high-resolution image-   Area 4: to process a high-resolution image

If the multiple-resolution image generation unit 11 generates threedifferent images, the image-process determination unit 13 determines asfollows:

-   Area 1: not to process a high-resolution image-   Area 2: to process a high-resolution image (medium)-   Area 3: to process a high-resolution image (medium)-   Area 4: to process a high-resolution image (high)

In this case, the image-process determination unit 13 defines theresolution process region to be a rectangle of 200 (horizontal)×100(vertical) pixels in the area 2, to be a rectangle of 150(horizontal)×75 (vertical) pixels in the area 3, and to be a rectangleof 100 (horizontal)×50 (vertical) pixels in the area 4.

<Variation 10: Defining Resolution Process Region in Accordance withTransport Task>

The image process-determination unit 13 determines whether or not thecurrent task executed by the robot R is a transport task. Now, the maincontroller 40 acquires a current task currently being executed from thetask information database 400 of the management computer 3, and outputsthe content (type) of this acquired task to the image-processdetermination unit 13. Alternatively, the internal condition detectionunit 45 outputs the task content stored in the status information to theimage-process determination unit 13. Based on the task content, theimage-process determination unit 13 determines whether or not the robotR is currently executing the transport task. Then, the image-processdetermination unit 13 defines the resolution process region on atransport container extracted by the low-resolution image process unit12.

As shown in FIG. 9, the robot R transports an article on a predeterminedtransport container N (tray). There may be provided a predeterminedpattern on the surface of the transport container N so that the robot Rcan easily recognize the image. FIG. 9 shows an example of a particularlogo provided on the transport container as the predeterminedrecognition pattern. However, the recognition pattern is not limited tothis, and may be any mark or pattern in any color which is differentfrom the base color of the transport container N. For the sake ofrecognition, the transport container N may preferably have a colordifferent from a surface color of a place where the transport containerN is placed. Details of such a transport of an article by the robot Rare disclosed in JP 2007-160447 A, for example.

<Variation 11: Defining Resolution Process Region in Accordance withGuide Task>

The image-process determination unit 13 determines whether or not thetask being currently executed by the robot R is a guide task, anddefines the resolution region depending on the direction of guiding aperson H. Now, the main controller 40 acquires the current task beingexecuted from the task information database 400 of the managementcomputer 3, and outputs the content of the acquired current task to theimage-process determination unit 13. Based on the acquired task content,the image-process determination unit 13 determines whether or not therobot R is currently executing the guide task. Then, as inputinformation, the image-process determination unit 13 defines theresolution process region at the guiding end point (see FIG. 4) includedin the input information that is stored in the current task. Details ofsuch a guide task by the robot R to guide the person H are disclosed inJP 2004-299026A, for example.

Returned to FIG. 6, the high-resolution image process unit 14 processesa region in the high-resolution image that corresponds to the resolutionprocess region at part of the low-resolution image defined by theimage-process determination unit 13, and generates target objectinformation regarding the target object included in the high-resolutionimage (also referred to as a “second target object information”).

Now, as similar to the low-resolution image process unit 12, thehigh-resolution image process unit 14 executes various steps ofgenerating parallax image, extracting a moving target object,recognizing a face region of a person, generating an edge-detected imageand generating a binary image, etc., so as to generate target objectinformation, which is output to the image-process result generation unit15.

The image-process result generation unit 15 determines whether or notthe target object information (first target object information)generated by the low-resolution image process unit 12 and the targetobject information (second target object information) generated by thehigh-resolution image process unit 14 are met. Based on thisdetermination, if both pieces of the target object information arematched, they are used to generate the recognition information. If bothpieces of the target object information are not matched, only the targetobject information generated by the high-resolution image process unit14 is used to generate the recognition information.

For instance, if the target object information generated by thelow-resolution image process unit 12 indicates: “there is an obstacle 5meters straight ahead of the robot R”, and target object informationgenerated by the high-resolution image process unit 14 indicates: “thereis an obstacle 10 meters straight ahead of the robot R”, this means thatboth pieces of the information are not matched. In such a case, theimage-process result generation unit 15 generates recognitioninformation by using only the target object information generated by thehigh-resolution image process unit 14. In this case, the image-processresult generation unit 15 generates such recognition information thatindicates: the type of the target object is an obstacle, the distance tothe target object is 10 meters, and the target object exists straightahead of the robot R (a direction of the target object relative to therobot R's heading direction), and outputs this generated recognitioninformation to the main controller 40.

Meanwhile, if the target object information of the low-resolution imageprocess unit 12 and the target object information of the high-resolutionimage process unit 14 are matched, the image-process result generationunit 15 uses both pieces of the target object information indicating theidentical content and outputs recognition information to the maincontroller 40.

If the image-process determination unit 13 determines not to process ahigh-resolution image, the image-process result generation unit 15 usestarget object information generated by the low-resolution image processunit 12, so as to generate recognition information, and outputs it tothe main controller 40. In this case, the recognition informationincludes variety of information regarding a target object, such as atype, direction and distance to a target object, a face region of aperson and a region of the target object in an image, etc.

[Audio Process Unit]

The audio processor 20 includes an audio synthesis unit 21 a, voicerecognition unit 21 b and sound-source position determination unit 21 c.Based on the instruction regarding speech behavior defined and output bythe main controller 40, the audio synthesis unit 21 a generates voicesound data from text data, and outputs voice sound based on thegenerated voice sound data through the speaker S.

Voice sound data is input through the microphones MC, MC to the voicerecognition unit 21 b (also referred to as “voice recognition means”),which generates text information from the input voice sound data, andoutputs this text information to the main controller 40.

The sound-source position determination unit 21 c identifies a positionof a sound source (a position in a plane state that the robot Rrecognizes) based on a difference in the sound pressure and the time ofsound arrival between the microphones MC, MC, and outputs thisidentified position of the sound source to the main controller 40. Theposition of the sound source may be represented by a rotational angle θzaround the direction where the robot R stands (i.e. z axis direction).

[Storage]

The storage 30 may be constituted, for example, of general-purposed harddisks, and stores necessary information (e.g. local map data and datafor speech) sent from the management computer 3. The storage 30 storesinformation necessary for the main controller 40 to execute variousoperations, as described later.

[Main Controller]

The main controller 40 integrally controls the image processor 10, theaudio processor 20, the storage 30, the autonomous motion control unit50, the wireless communication unit 60, the object detector 80 and thecircumference sensor 90.

Data detected by the gyro sensor SR1 and GPS receiver SR2 is output tothe main controller 40, where the received data is used fordetermination of the robot R's behavior. The main controller 40 performsvarious determinations to provide various controls, such as oncommunication to the management computer 3; on execution of apredetermined task in response to a task execution instruction acquiredfrom the management computer 3; on moving the robot R to a destination;on identification of a person: on conversation with a person, and alsogenerates various instructions for motions of each part of the robot R.

[Autonomous Motion Control Unit]

The autonomous motion control unit 50 drives the head R1, each arm R2,each leg R3 and the body R4, in response to the instructions from themain controller 40. The autonomous motion control unit 50 includes aneck control unit that drives a neck joint of the head R1; a handcontrol unit that drives finger joints of a hand of each arm R2; an armcontrol unit that drives a shoulder joints, elbow joints and wristjoints of the arms R2; a waist control unit that rotationally drives thebody R4 relative to the legs R3 in the horizontal direction; and a legcontrol unit that drives hip joints, knee joints and ankle joints of thelegs R3, which are not shown in the drawings. The neck control unit, thehand control unit, the arm control unit, the waist control unit and theleg control unit send their driving signals to the respective actuatorsthat drive the head R1, the arms R2, the legs R3 and the body R4,respectively. In addition, as mentioned above, the autonomous motioncontrol unit 50 may send an input traveling direction of the robot R oran input traveling speed of the robot R to the image-processdetermination unit 13, as input information, if necessary.

[Wireless Communication Unit]

The wireless communication unit 60 transmits and receives data to/fromthe management computer 3. The wireless communication unit 60 includes apublic line communication device 61 a and a wireless communicationdevice 61 b.

The public line communication device 61 a is a wireless communicationmeans utilizing public network such as mobile phone network, PHS(Personal Handyphone System) network. The wireless communication device61 b is a wireless communication means for a short distance wirelesscommunication such as wireless LAN complying with IEEE 802.11b.

In response to an access request from the management computer 3, thewireless communication unit 60 selects the public line communicationdevice 61 a or the wireless communication device 61 b to perform datacommunication with the management computer 3.

The battery 70 serves as a power supply source for supplying powerrequired for motion on each part of the robot R. A rechargeable batterymay be used as the battery 70, and recharging the battery 70 may becarried out at the battery supply areas B1, B2, B3 (see FIG. 1), forexample.

[Object Detector]

The object detector 80 detects whether or not a person wearing a tag Texists in the vicinity of the robot R. The object detector 80 includesplural light-emitting parts 81 (only one shown in FIG. 6). Thelight-emitting parts 81 may be constituted of, for example, LED, and maybe provided along the peripheral surface of the robot R's head R1, suchas on the right and left sides or the back and front sides thereof (notshown). The object detector 80 emits, from each light-emitting part 81,infra-red ray including a signal representing a light-emitting part IDto identify each light-emitting part 81, and then receives a receivingreport signal from the tag T when the gag T has received this infra-redray. When receiving infra-red ray emitted from any of the light-emittingparts 81, based on the light emitting part ID included in this receivedinfra-red ray, the tag T generates a receiving report signal; therefore,when the robot R refers to the light-emitting part ID included in thisreceipt report signal, the robot R determines in which direction the tagT exists, viewed from the robot R. In addition, the object detector 80has a function to determine the distance to the tag T, based on theradio field intensity of the receiving report signal acquired from thetag T. Hence, the object detector 80 can determine where the tag Texists (i.e. distance and direction of the tag T), which indicates aposition of the person H. Furthermore, the object detector 80 emits notonly infra-red ray from each light-emitting part 81, but also transmitsradio waves including a signal representing the robot ID from an antenna(not shown), whereby, when receiving the radio waves, the tag T cancorrectly identify which robot R has transmitted this infra-red ray.Details of such an object detector 80 and a tag T are disclosed inJP2006-192563A, for example.

[Circumference Sensor]

The circumference sensor 90 detects circumferential conditions in thevicinity of the robot R, and acquires self-position data detected by thegyro sensor SR1 or the GPS receiver SR2. The circumference sensor 90includes a laser radiation unit 91 that radiates a slit light toward asearch zone, a infra-red ray radiation unit 92 that radiates aninfra-red ray toward the search zone, and a floor-surface camera 93 thattakes an image of the search zone where the slit light or the infra-redray is radiated. The circumference sensor 90 analyses a slit-light image(image when the slit light was radiated) of the search zone that hasbeen taken by the floor-surface camera 93 and detects the floor surfaceconditions. In addition, the circumference sensor 90 analyzes theinfra-red ray image (when the slit light was radiated) taken by thefloor-surface camera 93 so as to detect the mark M (see FIG. 2), andbased on the position (coordinates) of the detected mark M, thecircumference sensor 90 calculates a positional relation between themark M and the robot R. Details of such a circumference sensor 90 aredisclosed in, for example, JP2006-167844A.

[Main Controller]

FIG. 10 is a block diagram showing the configuration of the maincontroller of the robot R in FIG. 6.

The main controller 40 includes a static obstacle integration unit 41,an object data integration unit 42, a behavior patterning unit 43, aninternal condition detection unit 45 and a behavior plan management unit46.

The static obstacle integration unit 41 integrates information regardingthe circumferential conditions in the vicinity of the robot R detectedby the circumference sensor 90, which is output to the behaviorpatterning unit 43. For example, when the static obstacle integrationunit 41 detects an obstacle, such as a cardboard container, or a step onthe floor surface of the traveling way of the robot R, and based on thisintegrated information regarding the obstacle, the behavior patterningunit 43 finds a detour route on a local detour module (not shown).

The object data integration unit 42 integrates identification data(object data) regarding an object, based on posture data of the robot R,recognition information generated by the image processor 10, and inputdata from the object detector 80 and the sound-source positiondetermination unit 21 c, and outputs this integrated object data to theobject data storing means 31 of the storage 30. Using this integratedobject data input, the object data storing means 31 creates an objectmap that records this integrated object data into the object type andthe time.

The behavior patterning unit 43 stores various programs (modules) toexecute an appropriate behavior pattern, and refers to the storage 30for getting information necessary when executing each behavior pattern,and reflects this necessary information in the behavior pattern.

In the present embodiment, as shown in FIG. 10, the storage 30 includesa local map data storing means 32 and a scenario storing means 33, aswell as an object data storing means 31.

The local map data storing means 32 stores maps of the vicinity of therobot R (local maps), as described with reference to FIG. 3. Forexample, the local maps may be acquired from the management computer 3.

The scenario storing means 33 stores various scenarios (acting scripts)correspondent to the respective behavior patterns. The scenarios includethose regarding actions of, for example, stopping 1 meter ahead of aperson or an obstacle (i.e. target object) when encountering this targetobject while walking, or lifting the arm R2 up to a predeterminedposition 10 minutes after stopping, as well as those regarding speech.The scenario storing means 33 also stores scenarios predefined forspecifying a gesture as a physical behavior of moving at least one ofthe head R1, the arms R2, the legs R3 and the body R4 when the robot Rperforms a predetermined speech.

The behavior patterning unit 43 includes various modules that executecorrespondent behavior patterns in accordance with variety of scenes andsituations, using the object data storing means 31, the local map datastoring means 32 or the scenario storing means 33, or in combinationtherewith, if necessary. The various modules may be, for example, adestination path module, a local detour module, a delivery module, aguide module and a human handling module, etc.

The destination path module finds a path from the robot R's currentpoint to a destination where the robot R executes a particular task inthe task execution area (e.g. searching a path between the nodes), andexecutes a traveling action along the found path to the destination.This destination path module refers to the map data and the currentpoint of the robot R, and then calculates a minimum distance to thedestination.

When an obstacle is detected while walking, the local detour modulefinds a detour route to get around the detected obstacle based on theobstacle information integrated by the static obstacle integration unit41.

When an article delivery task is executed, the delivery module performsan action of receiving (gripping) an article from a person (client) whorequests the article delivery, or an action of handing over (releasing)the article to a receiving person.

The guide module, for example, executes a task to navigate a visitor whocame to a guide start point in the task execution area to a clerk at theinformation counter 305 in the guide area 301 (see FIG. 3).

When the article delivery task or the guide task is executed, the humanhandling module, for example, performs an action of speech, posturechange, moving the arm R2 up and down or gripping, etc.

The internal condition detection unit 45 detects internal conditions ofthe robot R. In the present embodiment, the internal condition detectionunit 45 detects the remaining power of the battery 70, for example. Theinternal condition detection unit 45 generates data regarding conditionsof the robot R (e.g. the current position, the remaining power of thebattery, the task execution status, etc.) as the status information atthe predetermined time intervals, and outputs the generated statusinformation to the conversation-triggering action control unit 47. Theinternal condition detection unit 45 outputs the generated statusinformation via the wireless communication unit 60 to the managementcomputer 3. Then, the management computer 3 registers for each robot Rthe input status information on the robot information database (notshown) stored in the storage 3 a (see FIG. 1).

The behavior plan management unit 46 manages behavior plans to executethe various modules of the behavior patterning unit 43 in accordancewith predetermined schedules. In the present embodiment, the behaviorplan management unit 46 manages behavior plans, so as to execute aappropriate task in response to task execution instructions acquiredfrom the management computer 3, and select an appropriate modulerequired for executing the current task to be done.

[Image Processor Operations]

FIG. 11 is a flow chart showing various steps performed by the imageprocessor 10 of FIG. 6

On the multiple-resolution image generation unit 11, the image processor10 corrects distortions of an original image acquired by the visualsensor (S1), and then the image processor 10 down-sizes thisdistortion-corrected image to generate a lower resolution image having alower resolution than that of the original image, and also generates atleast one high-resolution image having a higher resolution than that ofthe low-resolution image (S2). On the image-process determination unit13, the image processor 10 determines whether or not to process thehigh-resolution image in accordance with a predetermined rule or giveninput information (S3).

If the image-process determination unit 13 determines to process thehigh-resolution image (Yes at S3), the image processor 10, on thelow-resolution image process unit 12, processes the low-resolution imagegenerated by the multiple-resolution image generation unit 11, andgenerates target object information regarding a target object includedin the low-resolution image (S4). On the image-process determinationunit 13, the image processor 10 determines which high-resolution imageto be processed if the multiple-resolution image generation unit 11generates more than one high-resolution image, and also define aresolution process region on part of the low-resolution image (S5). Onthe high-resolution image process unit 14, the image processor 10processes the resolution process region included in the high-resolutionimage, defined by the image-process determination unit 13, and generatesthe target object information regarding the target object included inthe high-resolution image (S6).

If it is determined not to process the high-resolution image on theimage-process determination unit 13, the image processor 10, on thelow-resolution image process unit 12, processes the low-resolution imagegenerated by the multiple-resolution image generation unit 11, and thengenerates the target object information regarding the target objectincluded in the low-resolution image (S8). In this case, theimage-process determination unit 13 of the image processor 10 defines noresolution process region, and the high-resolution image process unit 14of the image processor 10 generates no target object information.

Following S6, on the image-process result generation unit 15, the imageprocessor 10 determines whether or not the target object information ofthe low-resolution image process unit 12 and the target objectinformation of the high-resolution image process unit 14 are matched. Ifboth pieces of the information are matched, the image processor 10 usesboth pieces of the target object information, otherwise the imageprocessor 10 uses the target object information of the high-resolutionimage process unit 14 only, so as to generate recognition information,which is output to the main controller 40 (S7).

If following the S8, the image processor 10 generates, on theimage-process result generation unit 15, the recognition informationusing the target object information generated by the low-resolutionimage process unit 12, and outputs this generated recognitioninformation to the main controller 40.

As explained above, according to the present embodiment, theimage-process determination unit 13 determines whether or not to processan image of interest, and if determining to process the image, theimage-process determination unit 13 also appropriately determines whichresolution of the image to be processed, and also defines a resolutionprocess region for this image. Accordingly, the robot R of the presentinvention can increase the accuracy in recognition of a target objectwhile reducing calculations required for the image process. Although thepreferred embodiment of the present invention has been explained, thepresent invention is not limited to this explained embodiment. Forexample, in the preferred embodiment, the determining whether or not toprocess an image of interest on the image-process determination unit 13,and shapes and sizes of a resolution process region are merelyexemplified for a good understanding of the present invention.

In the present embodiment, the robot R is explained as a two-leg typeautonomous mobile robot, but is not limited to this, and may beapplicable to other type robots such as an autonomous mobile robot thatmoves with wheels. In this case, such an autonomous mobile robot,although having movable members of wheels that correspond to legs of atwo-leg type autonomous robot, can achieve effects equivalent to thoseby the present invention.

The mobile robot of the invention provides the following effects.

In the mobile robot according to the present embodiment, thehigh-resolution image process unit defines a resolution process regionbased on an input information or a predetermined rule. Accordingly, themobile robot is capable of desirably handling various situationsincluding movements and or tasks performed by the robot, or targetobjects existing in the vicinity of the robot. In the mobile robotaccording to the present embodiment, the high-resolution image processunit processes only part of a high-resolution image having higherresolution. Accordingly, the mobile robot secures of the presentembodiment secures a higher accuracy in recognition of a target object,while reducing calculations required for the image process.

In addition, the mobile robot according to the present embodimentsecures a higher accuracy in recognition of a target object which islocated straight ahead of the robot or in the direction of the robot'shead facing.

Furthermore, the mobile robot according to the present embodiment alsosecures a higher accuracy in recognition of a target object, such as aperson, which moves in the horizontal direction.

The embodiment according to the present invention has been explained asaforementioned. However, the embodiment of the present invention is notlimited to those explanations, and those skilled in the art ascertainthe essential characteristics of the present invention and can make thevarious modifications and variations to the present invention to adaptit to various usages and conditions without departing from the spiritand scope of the claims.

1. A mobile robot comprising an image processor that processes an imagetaken by a vision sensor and generates recognition information regardinga target object to be recognized included in the taken image, and acontroller that integrally controls the mobile robot based on thegenerated recognition information, the image processor comprising: amultiple-resolution image generation unit that down-sizes the takenimage to generate a low-resolution image having a resolution lower thanthat of the taken image and at least one high-resolution image having aresolution higher than that of the low-resolution image; alow-resolution image process unit that processes the low-resolutionimage generated by the multiple-resolution image generation unit, andgenerates first target object information regarding the target objectincluded in the low-resolution image; an image-process determinationunit that determines whether or not to process the high-resolution imagein accordance with predetermined input information or a predeterminedrule, also determines which high-resolution image having what resolutionshould be processed if the multiple-resolution image generation unitgenerates two or more high-resolution images, and then defines aresolution process region at part of the low-resolution image; ahigh-resolution image process unit that processes a region in thehigh-resolution region that corresponds to the resolution process regionat part of the low-resolution image defined by the image-processdetermination unit so as to generate second target object informationregarding the target object included in the high-resolution image; andan image-process result generation unit that: determines whether or notthe first target object information generated by the low-resolutionimage process unit and the second target object information generated bythe high-resolution image process unit are met, and based on thedetermination of the first and second target object information, uses atleast either of the first and the second target object information,first and the second target object information, thereby to generate therecognition information.
 2. The mobile robot according to claim 1,wherein, in accordance with the predetermined rule, the image-processdetermination unit defines the resolution process region at a center ofthe low-resolution image.
 3. The mobile robot according to claim 1,wherein, in accordance with the predetermined rule, the image-processdetermination unit repeatedly defines the resolution process region inthe low-resolution image, shifting the resolution process region in thehorizontal direction in the low-resolution image.
 4. The mobile robotaccording to claim 1, wherein the low-resolution image process unitextracts a face region of a person included in the low-resolution imageas input information, and the image-process determination unit definesthe resolution process region on this extracted face region extracted bythe low-resolution image process unit.
 5. The mobile robot according toclaim 1, wherein, if input traveling direction or a future travelingdirection of the mobile robot is input to the image-processdetermination unit as the input information, the image-processdetermination unit defines the resolution process region in thelow-resolution image depending on the input traveling direction or thefuture traveling direction of the mobile robot.
 6. The mobile robotaccording to claim 1, wherein, if input traveling speed or a futuretraveling speed of the mobile robot is input to the image-processdetermination unit as the input information, the image-processdetermination unit is more likely to determine to process ahigher-resolution image and narrows the resolution process region in thelow-resolution image as the input traveling speed or the futuretraveling speed becomes faster.
 7. The mobile robot according to claim1, wherein, if a direction and a sound pressure of a sound source of thetarget object are input as the input information from a sound-sourceposition determination unit that identifies the direction and the soundpressure of the sound source, the image-process determination unitdefines the resolution process region in the low-resolution imagedepending on the direction of the sound source.
 8. The mobile robotaccording to claim 7, wherein, if the input sound pressure becomessmaller, the image-process determination unit is more likely todetermine to process a higher-resolution image and narrows theresolution process region in the low-resolution image.
 9. The mobilerobot according to claim 1, wherein, if a direction and a distance to atag attached on an object to be detected are input from a objectdetector for detecting the direction and the distance to the tag, theimage-process determination unit defines the resolution process regionin the low-resolution image depending on the input direction of the tag.10. The mobile robot according to claim 9, wherein the image-processdetermination unit is more likely to determine to process ahigher-resolution image and narrows the resolution process region in thelow-resolution image as the input distance of the tag becomes greater.11. The mobile robot according to claim 1, wherein, when the mobilerobot executes a transport task to transport an article, thelow-resolution image process unit extracts a predetermined transportcontainer on which the article are placed, which is included in thelow-resolution image, as input information, the image-processdetermination unit determines whether or not a task being currentlyexecuted by the mobile robot is the transport task, and defines theresolution process region on the transport container extracted by thelow-resolution image process unit.
 12. The mobile robot according toclaim 1, wherein, when the mobile robot executes a guide task to guide aperson, a guide direction in which the person should be guided is inputto the image-process determination unit as input information, theimage-process determination unit determines whether or not a task beingcurrently executed by the mobile robot is the guide task, and definesthe resolution process region in the low-resolution image depending onthe input guide direction.
 13. The mobile robot according to claim 1,wherein the image-process result generation unit uses both the first andthe second target object information if the first and the second targetobject information are matched; otherwise uses only the second targetobject information if not matched, thereby to generate the recognitioninformation.